[Maps] Do not fetch geo_shape from docvalues #63997

nreese · 2020-04-20T19:55:15Z

geo_points are fetched from doc_values to get a consistently formatted value. The logic is written very generically so that any field that supports doc values will be fetched with doc values.

Elasticsearch has recently merged support for geo_shape docvalues (elastic/elasticsearch#55037). Looks like there is a bug when requesting doc values for geo_shape fields. This is just a quick fix to not request doc values until the bug for geo_shapes gets figured out.

Example data set with geo_shape field.

PUT /example
{
    "mappings": {
        "properties": {
            "location": {
                "type": "geo_shape"
            }
        }
    }
}
POST /example/_doc
{
    "location" : {
        "type" : "linestring",
        "coordinates" : [[-77.03653, 38.897676], [-77.009051, 38.889939]]
    }
}

Working query

GET example/_search
{
  "query": {
    "match_all": {}
  }
}

Requesting docvalue_fields fails.

GET shapes/_search
{
  "query": {
    "match_all": {}
  },
  "docvalue_fields": ["location"]
}

{
  "error" : {
    "root_cause" : [
      {
        "type" : "unsupported_operation_exception",
        "reason" : "scripts and term aggs are not supported by geo_shape doc values"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "example",
        "node" : "Ef8381lmQ82FBAh45eWkZA",
        "reason" : {
          "type" : "unsupported_operation_exception",
          "reason" : "scripts and term aggs are not supported by geo_shape doc values"
        }
      }
    ],
    "caused_by" : {
      "type" : "unsupported_operation_exception",
      "reason" : "scripts and term aggs are not supported by geo_shape doc values",
      "caused_by" : {
        "type" : "unsupported_operation_exception",
        "reason" : "scripts and term aggs are not supported by geo_shape doc values"
      }
    }
  },
  "status" : 500
}

elasticmachine · 2020-04-20T19:55:17Z

Pinging @elastic/kibana-gis (Team:Geo)

thomasneirynck

Thank you for the quick turn-around.

Can you link the ES bug report here too? Ideally, this gets returned from ES as expected.

nreese · 2020-04-20T20:05:26Z

opened elastic/elasticsearch#55487 to track issue on Elasticsearch side

kibanamachine · 2020-04-20T21:33:37Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 4b21cda

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* master: (30 commits) Migrate vis_type_table to kibana/new platform (elastic#63105) Enable include/exclude in Terms agg for numeric fields (elastic#59425) follow conventions for saved object definitions (elastic#63571) [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935) kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838) Fix page layouts, clean up unused code (elastic#63992) [SIEM] Adds recursive exact key checks for validation and formatter [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859) [Maps] Do not fetch geo_shape from docvalues (elastic#63997) Vega doc changes (elastic#63889) [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049) Add sub-menus to Resolver node (for 75% zoom) (elastic#63476) [FTR] Add test suite metrics tracking/output (elastic#62515) [ML] Fixing single metric viewer page padding (elastic#63839) [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623) [Reporting] Config flag to escape formula CSV values (elastic#63645) [Metrics UI] Remove remaining field filtering (elastic#63398) [Maps] fix date labels (elastic#63909) [TSVB] Use default Kibana palette for split series (elastic#62241) Ingest overview page (elastic#63214) ...

…t-node-pipelines * 'master' of github.com:elastic/kibana: (32 commits) Move authz lib out of snapshot restore (#63947) Migrate vis_type_table to kibana/new platform (#63105) Enable include/exclude in Terms agg for numeric fields (#59425) follow conventions for saved object definitions (#63571) [Docs]Adds saved object key setting to load balancing kib instances (#63935) kbn/config-schema: Consider maybe properties as optional keys in ObjectType (#63838) Fix page layouts, clean up unused code (#63992) [SIEM] Adds recursive exact key checks for validation and formatter [Maps] Migrate remaining maps client files to NP (except routi… (#63859) [Maps] Do not fetch geo_shape from docvalues (#63997) Vega doc changes (#63889) [Metrics UI] Reorganize file layout for Metrics UI (#60049) Add sub-menus to Resolver node (for 75% zoom) (#63476) [FTR] Add test suite metrics tracking/output (#62515) [ML] Fixing single metric viewer page padding (#63839) [Discover] Allow user to generate a report after saving a modified saved search (#63623) [Reporting] Config flag to escape formula CSV values (#63645) [Metrics UI] Remove remaining field filtering (#63398) [Maps] fix date labels (#63909) [TSVB] Use default Kibana palette for split series (#62241) ...

…bana into ingest-node-pipelines/privileges * 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits) Move authz lib out of snapshot restore (elastic#63947) Migrate vis_type_table to kibana/new platform (elastic#63105) Enable include/exclude in Terms agg for numeric fields (elastic#59425) follow conventions for saved object definitions (elastic#63571) [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935) kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838) Fix page layouts, clean up unused code (elastic#63992) [SIEM] Adds recursive exact key checks for validation and formatter [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859) [Maps] Do not fetch geo_shape from docvalues (elastic#63997) Vega doc changes (elastic#63889) [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049) Add sub-menus to Resolver node (for 75% zoom) (elastic#63476) [FTR] Add test suite metrics tracking/output (elastic#62515) [Ingest pipelines] Delete pipeline (elastic#63635) [ML] Fixing single metric viewer page padding (elastic#63839) [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623) [Reporting] Config flag to escape formula CSV values (elastic#63645) [Metrics UI] Remove remaining field filtering (elastic#63398) [Maps] fix date labels (elastic#63909) ... # Conflicts: # x-pack/legacy/plugins/uptime/public/components/monitor/ml/index.ts # x-pack/legacy/plugins/uptime/public/components/overview/index.ts # x-pack/plugins/ingest_pipelines/public/application/index.tsx # x-pack/plugins/ingest_pipelines/server/routes/api/index.ts # x-pack/plugins/ingest_pipelines/server/routes/index.ts

…bana into pipeline-editor-part-mvp-2 * 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits) [Ingest Node Pipelines] Clone Pipeline (elastic#64049) Move authz lib out of snapshot restore (elastic#63947) Migrate vis_type_table to kibana/new platform (elastic#63105) Enable include/exclude in Terms agg for numeric fields (elastic#59425) follow conventions for saved object definitions (elastic#63571) [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935) kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838) Fix page layouts, clean up unused code (elastic#63992) [SIEM] Adds recursive exact key checks for validation and formatter [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859) [Maps] Do not fetch geo_shape from docvalues (elastic#63997) Vega doc changes (elastic#63889) [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049) Add sub-menus to Resolver node (for 75% zoom) (elastic#63476) [FTR] Add test suite metrics tracking/output (elastic#62515) [Ingest pipelines] Delete pipeline (elastic#63635) [ML] Fixing single metric viewer page padding (elastic#63839) [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623) [Reporting] Config flag to escape formula CSV values (elastic#63645) [Metrics UI] Remove remaining field filtering (elastic#63398) ...

[Maps] Do not fetch geo_shape from docvalues

4b21cda

nreese added [Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.8.0 labels Apr 20, 2020

nreese requested a review from thomasneirynck April 20, 2020 19:55

nreese requested a review from a team as a code owner April 20, 2020 19:55

thomasneirynck approved these changes Apr 20, 2020

View reviewed changes

nreese merged commit 35dab18 into elastic:master Apr 20, 2020

nreese mentioned this pull request Apr 20, 2020

[7.x] [Maps] Do not fetch geo_shape from docvalues (#63997) #64013

Merged

nreese added a commit to nreese/kibana that referenced this pull request Apr 20, 2020

[Maps] Do not fetch geo_shape from docvalues (elastic#63997)

c6afc0f

nreese added a commit that referenced this pull request Apr 21, 2020

[Maps] Do not fetch geo_shape from docvalues (#63997) (#64013)

98369bb

nreese mentioned this pull request Apr 22, 2020

Set readFromDocValues to false for geo_shape fields #64014

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Maps] Do not fetch geo_shape from docvalues #63997

[Maps] Do not fetch geo_shape from docvalues #63997

nreese commented Apr 20, 2020 •

edited

Loading

elasticmachine commented Apr 20, 2020

thomasneirynck left a comment

nreese commented Apr 20, 2020

kibanamachine commented Apr 20, 2020

[Maps] Do not fetch geo_shape from docvalues #63997

[Maps] Do not fetch geo_shape from docvalues #63997

Conversation

nreese commented Apr 20, 2020 • edited Loading

elasticmachine commented Apr 20, 2020

thomasneirynck left a comment

Choose a reason for hiding this comment

nreese commented Apr 20, 2020

kibanamachine commented Apr 20, 2020

💚 Build Succeeded

nreese commented Apr 20, 2020 •

edited

Loading